Today I found out that the DNS-Server at home is used for a DDoS against a DNS-Server. The attackers send a small query, "ripe.net ANY" in this case, with a faked sender IP. The DNS-Server answers with a much larger packet to the (faked) sender. Because the domain is not local, the nameserver should have only produced an error. But the bind-config allowed everyone to get answers to cached answers:

allow-query-cache   0.0.0.0/0; ::/0;  ;

With proper restriction, the server only returns errors now. Restricting answer for cached entries does not help if the queried nameserver is authoritative for the domain. In this case it can help to drop queries for the ANY type. In ferm this looks like this:

proto udp dport domain mod string from 40 algo bm hex-string ' 0000ff0001 ' DROP;

I installed a Lenovo Thinkpad X130e with Wheezy.

AoE is one of the protocols supported by Linux to access storage via network. It uses plain Ethernet for communication and includes a discovery mechanism to find all available targets. I use it to provide disk space to VMs running on different machines. Next step is to boot via AoE. It is no real problem to use AoE in running systems. However with some help it is even possible to actually boot disk-less machines via AoE. The PXE implementation iPXE provides the AoE support to actually boot from. I will describe the necessary parts.

The statistics of the "who wrote Linux x.y.z" series date back to at least 2.6.20. According to my experience talking with users and Free Software enthusiasts, those statistics really make a dent in the public perception of who is giving back upstream. Obviously, one should not take a single upstream, even if it is as important as the Linux kernel, as a measure of how much a given Free Software entity is giving back upstream overall. But users still seem to be fascinated by them. As a result, I have often had to answer the question: why Debian doesn't show up on those statistics?. My answer has always been something along the lines that Debian Developers who maintain Linux kernel packages, the almighty Debian Kernel Team, do that mostly as part of their volunteer engagement in Debian. As a consequence, they do not earmark their contributions as if they worked for a company and they add up to the hobbyist count instead (although you can you can routinely spot individual Debian Kernel Team members among the most active contributors for specific Linux releases). The above is the true and honest answer. But every time I've given it, I couldn't help feeling that the user who asked went home with a "yeah, well" afterthought. If you don't want to take my word of it, fine. Here is what Greg K-H had to say about Debian contributions in a recent blog post about the stable Linux kernel:

I would personally like to thank the Debian kernel developers, specifically Ben Hutchings, Maximilian Attems, Dann Frazier, Bastian Blank, and Moritz Muehlenhoff. They went above and beyond what any "normal" developer would have done, ferreting patches out of the kernel.org releases and the different vendor kernels and bug tracking systems, backporting them to the 2.6.32 kernel, testing, and then forwarding them on to me. Their dedication to their user community is amazing for such a "volunteer" group of developers. I firmly believe that without their help, the 2.6.32 kernel would not have been the success that it was. The users of Red Hat and SuSE products owe them a great debt. Buy them a beer the next time you see them, they more than deserve it.

I'll take good care of following his wise advice. Please do the same.

---

(Thanks to Sylvestre for pointing me to Greg's blog post.)

For convenience, I want to store copies of all my DVDs on my file server. It allow me to save shelf space flat while still having my movie collection easily available. It also make it possible to let the kids see their favourite DVDs without wearing the physical copies down. I prefer to store the DVDs as ISOs to keep the DVD menu and subtitle options intact. It also ensure that the entire film is one file on the disk. As this is for personal use, the ripping is perfectly legal here in Norway. Normally I rip the DVDs using dd like this:

#!/bin/sh
# apt-get install lsdvd
title=$(lsdvd 2>/dev/null awk '/Disc Title: /  print $3 ')
dd if=/dev/dvd of=/storage/dvds/$title.iso bs=1M

But some DVDs give a input/output error when I read it, and I have been looking for a better alternative. I have no idea why this I/O error occur, but suspect my DVD drive, the Linux kernel driver or something fishy with the DVDs in question. Or perhaps all three. Anyway, I believe I found a solution today using dvdbackup and genisoimage. This script gave me a working ISO for a problematic movie by first extracting the DVD file system and then re-packing it back as an ISO.

#!/bin/sh
# apt-get install lsdvd dvdbackup genisoimage
set -e
tmpdir=/storage/dvds/
title=$(lsdvd 2>/dev/null awk '/Disc Title: /  print $3 ')
dvdbackup -i /dev/dvd -M -o $tmpdir -n$title
genisoimage -dvd-video -o $tmpdir/$title.iso $tmpdir/$title
rm -rf $tmpdir/$title

Anyone know of a better way available in Debian/Squeeze? Update 2011-09-18: I got a tip from Konstantin Khomoutov about the readom program from the wodim package. It is specially written to read optical media, and is called like this: readom dev=/dev/dvd f=image.iso. It got 6 GB along with the problematic Cars DVD before it failed, and failed right away with a Timmy Time DVD. Next, I got a tip from Bastian Blank about his program python-dvdvideo, which seem to be just what I am looking for. Tested it with my problematic Timmy Time DVD, and it succeeded creating a ISO image. The git source built and installed just fine in Squeeze, so I guess this will be my tool of choice in the future.

During Debconf 11, I got access to a fast s390 machine, and I have started to work on a Debian s390x port, the 64-bit version of the s390 port. One of my goal was to help the SPARC64 port, as some of the issues are the same: both are 64-bit big-endian, don t support unaligned access and behave differently between -fpic and -fPIC. Why such a port? When talking about 64-bit ports, we usually hear: 4GB is enough, handling 64-bit takes more memory . This really sounds like 640K ought to be enough for anybody . The s390 port is actually 31-bit from the address point of view (one bit is reserved for address space extension from 24 to 31 bits), so each process is limited to 2GB only. Nowadays applications which need more than 2GB are not that uncommon, especially on mainframes. Actually the 2GB limit already causes some problem in Debian: in some cases it s not possible to build haskell applications or even C applications using GCC. On the other hand, we already require a 64-bit kernel on the s390 port (only the userland is 32-bit), and applications are handling more and more 64-bit or greater values (files offset, time counters, uid, etc.). What is the status? Bootstrapping the architecture was not really easy (as for any other new architectures), due to a huge amount of dependencies and build-dependencies loops, as explained by Wookey during Debconf11. Now that this part is mostly done, an autobuilder has been started and currently more than 65% of the packages are built. The s390x port is hosted on debian-ports.org. Unfortunately it is not yet deboostrapable, though that should happen in the next few days (only a few packages are missing). The main issues are currently packages which fail to build from source due to linker, gcc-4.6 and curl changes, or due to the libjpeg and multiarch transitions, and thus are not directly related to s390x. If your package is in this case, it would be a good idea to fix it. Otherwise if it has a lot of reverse dependencies and the bug is opened for a while, just expect an NMU (as allowed by the 0-day NMU policy). Of course for a few packages s390x specific fixes are needed, some of them are already in the BTS. How you can help? The list of bugs blocking the s390x port is available through the s390x usertag, fixing these bugs (a lot of them are general FTBFS) would help a lot. Alternatively if you have access to an s390x machine you can take a look at the packages failing to build. Update: Fixed the explanation about the 32th bit, thanks to Bastian Blank for the comment.

It took a long time to get all the parts of the Xen support into the Linux kernel. While rudimentary Dom0-support was available since 2.6.38, support for device backends were missing. It was possible to replace this backend with a userspace implementation included in qemu, but I never tested that. With Linux 3.0, both the traditional block backend and the network backend are available. They are already enabled in the current 3.0-rc3/-rc4 packages in experimental, so the packages can be used as Dom0 and run guests. Right now the backend modules are not loaded, so this still needs some work. Neither the init scripts loads them, because the names where in flux the last time I laid hand on it, nor does the kernel themself expose enough information to load them via udev. I think using udev to load the modules is the way to go. This step marks the end of a five year journey. Around 2.6.16 the Xen people started to stay really close to Linux upstream. With the 2.6.18 releas this stopped and the tree was pushed in different states into Debian Etch and RHEL 5. After that, Xen upstream ceased work on newer versions completely, only changes to the now old 2.6.18 tree where done. SuSE started a forward port of the old code base to newer kernel versions and Debian Lenny released with such a patched 2.6.26. Around that time, minimal support for DomU on i386 using paravirt showed up and Lenny had two different kernels with Xen support. Since 2.6.28 this support was mature and works rather flawless since. Somehow after that, a new port of the Dom0 support, now using paravirt, showed up. This tree based on 2.6.32 is released with Debian Squeeze. After several more rounds of redefining and polishing it is now mostly merged into the core kernel. I don't know what the future brings. We have two virtualization systems supported by Linux now. The first is KVM that converts the kernel into a hypervisor and runs systems with help of the hardware virtualization. The later one is Xen that runs under a standalone hypervisor and supports both para- and hardware virtualization. Both works, KVM is easier to use and even works on current System z hardware. It can be used by any user with hopefully enough margin of security between them. Xen's home is more suited for servers, where you don't have users at all. Both have advantages and disadvantages, so everyone have to decide what he needs, there is no "one size fits all".

Recently I read Alexandre Rosenfeld s blog post about low performance of btrfs while placing VM s disk on it, so I had a try and got almost the same result. I tried to have someone look into the problem, and now get the answer, quoting Bastian Blank : This is a result of the filesystem design, [...]

After a long time, I decided to write again. I decided to start with software I wrote for my own usage that could be usefull for other people. I'll start with python-dvdvideo, a DVD-Video reader written in Python 3, and the reference tool dvd-video-backup-image, a generic DVD-Video dumper. Lets see, if this blog will see more postings in the future.

A few of us are working on spam removal from Debian lists archives. The wiki page linked above explains how to report spam on Debian mailing lists. This is in short as easy as bouncing a mail to a specific address, from your favourite MUA. These "reported spams" then need to be reviewed. Once a given message has been identified as "spam" by enough DD's (there are many false positives in the candidates, particularly in non English-speaking mailing lists), it is removed from the archives. Many mails have already been removed and any help is welcomed. Since Frans Pop launched this for debian-boot, back in May, I use 1 or 2 hours every Sunday to this work. After working on debian-boot only, I gradually worked on reported spams in other lists. As of now, I only have 4 lists where I still have work to do:

• debian-user (115 of 2420 reviewed)
• debian-user-spanish (177 of 1157 reviewed)
• debian-devel (136 of 1075 reviewed)
• debian-chinese-gb (159 of 556 reviewed)

The Chinese and the Spanish ones are tricky because identifying spam there is much less easy (for Chinese, I'm quite conservative and only tag very obvious spam....for Spanish, I read enough of the language to be able to target spam). What about you? Will you be able to help the few of us who work on getting clean archives (noticeably, Sandro Tosi, Giacomo Catenazzi, Cord Beermann, Luk Claes, Frans Pop, Bastian Blank, Luca Falavigna, Michael Koch, Bernd Steinmetz, Thoomas Viehmann, Florian Ernst, Adam D. Barratt, to name thos ewho reviewed more than 1000 mails)? Working on lists in your own language might be a good idea (I'm particularly thinking about lists in German, Spanish, Chinese and French).

I was asked to take a look at a machine where aptitude don't even want to do the upgrade to etch. A first inspection shows some weird repositories in the source.list file and many daemons noone ever should use on that machine. I was able to do the upgrade with apt-get then. After some time I got asked over modifications in /etc/exports. It basically included the following content:[1]

/     *(rw,async,no_root_squash)
/home *(rw,async,no_root_squash,nohide)
/usr  *(rw,async,no_root_squash,nohide)
/var  *(rw,async,no_root_squash,nohide)

[1]	For those who don't speak NFS: This exports the specified filesystems (/, /home, /usr and /var) to everyone, and accepts whatever the client system say.

The machine was taked out of service immediately. We'll have no chance to answer the question whether this was silliness or intend.

Today I asked if it is possible to do automatic updates of the version in metadata.xml from the (possible mangled) version from setup.py. Nothing popped up, so I wrote an extension of setuptools which does this.

import os.path
from distutils import log
from setuptools.command.egg_info import egg_info as _egg_info
class egg_info(_egg_info):
    def run(self):
        _egg_info.run(self)
        version = self.distribution.metadata.version
        for package in self.distribution.packages:
            path = os.path.join(*(package.split('.') + ['profiles', 'default']))
            if os.path.isdir(path):
                metadata_out = os.path.join(path, 'metadata.xml')
                metadata_in = os.path.join(path, 'metadata.xml.in')
                if os.path.exists(metadata_in):
                    log.info('writing %s', metadata_out)
                    d = open(metadata_in, 'r').read().replace('@VERSION@', version)
                    open(metadata_out, 'w').write(d)
setup(
    cmdclass =  'egg_info': egg_info ,
    [...]
)

The version is listed with a placeholder in the input file called metadata.xml.in and is replaced during a normal develop call as done by buildout.

<metadata>
    <version>@VERSION@</version>
</metadata>

UPDATE: This is evil. metadata.xml should list the the config/profile version, not the code version.

I was forced to try Ubuntu Hardy in the new university pool. The setup includes one Linux server dedicated for the pool, one Windows AD for Kerberos authentication, one Windows fileserver with user data and 20 clients. The clients are new HP machines with a Radeon Xpress 200 card. First problem was nasty, the X server turned the display black and then crashed, leaving an unusable console behind. Even a blacklist of the radeon module does not work. Somehow the Xorg radeon driver loads the module on its own, ignoring the modprobe blacklist. Only a hard blacklist using install radoen /bin/false in the modprobe config was able to prevent this. Lets hope that the endeavors to remove much priviledges from the X server goes good. For the homes two setups was tested:

• Homes on the Windows server via cifs, mounted via pam_mount using NTLM password.
• Homes on the Linux server via nfs version 3.

The first one just produced an error that some Gnome component was unable to lock ~/.ICEauthority. This could be worked around in the Xsession. Otherwise it was usable to slow, which may be a problem with the Windows server. The later, plus a reinstallation with fglrx, produced first unresponsive OpenOffice.org windows and then a complete unusable Gnome desktop. I would not completely vote against a problem with fglrx or even a broken installation, but the amount of problems exceeded the threshold. Now we will try Lenny and see if this also happens there. Especially home via nfs is not that uncommon that we can let it broken in a release.

I found a machine which is not so ancient and did some tests with Xen on it.

I found some weird opcodes in the s390 instruction set some time ago. I finally want to use it as I have access to such a machine. The result is rather slim:

.globl cu41
        .type   cu41, @function
cu41:
.L2:
        cu41    %r2,%r4
        ipm     %r1
        srl     %r1,28
        chi     %r1,3
        je      .L2
        ltr     %r1,%r1
        je      .L3
        lcr     %r1,%r1
        lgfr    %r3,%r1
.L3:
        lgr     %r2,%r3
        br      %r14

The cu41 opcode translates UTF32 to UTF8. Each "parameter" is a register pair [1], which describes address and length of the buffer. This opcode will return after a not specified number of translated characters. The rest is boiler plate code for condition code checking. The loop makes sure that anything is converted within the function unless another error occured.

[1]	An even pair. gcc is not yet able to allocate such pairs on its own, so it is hardcoded.

I found some weird opcodes in the s390 instruction set some time ago. I finally want to use it as I have access to such a machine. The result is rather slim:

.globl cu41
        .type   cu41, @function
cu41:
.L2:
        cu41    %r2,%r4
        ipm     %r1
        srl     %r1,28
        chi     %r1,3
        je      .L2
        ltr     %r1,%r1
        je      .L3
        lcr     %r1,%r1
        lgfr    %r3,%r1
.L3:
        lgr     %r2,%r3
        br      %r14

The cu41 opcode translates UTF32 to UTF8. Each "parameter" is a register pair [1], which describes address and length of the buffer. This opcode will return after a not specified number of translated characters. The rest is boiler plate code for condition code checking. The loop makes sure that anything is converted within the function unless another error occured.

[1]	An even pair. gcc is not yet able to allocate such pairs on its own, so it is hardcoded.

Xen 3.3 added a tool called PV-GRUB. It is a GRUB (legacy aka 0.97) built against MiniOS and bootable as a PV kernel. It works fine on a Xen 3.2.1. It is easy to use, I just set this in my config:

kernel = "/usr/local/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

However it decided to not load my config as defined in the config and simply show me that:

    GNU GRUB  version 0.97  (524288K lower / 0K upper memory)
       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]
grubdom>

The reason is quite easy:

grubdom> root (hd0)
 Filesystem type unknown, using whole disk

Some debugging later, the reason is known and fixed.

grubdom> root (hd0)
 Filesystem type is ext2fs, using whole disk
grubdom> kernel /boot/vmlinuz-2.6.27-rc5-amd64
grubdom> initrd /boot/initrd.img-2.6.27-rc5-amd64
grubdom> boot
block error -1 for op 2
close blk: backend at /local/domain/0/backend/vbd/23/51712
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.27-rc5-amd64 [...]
[    0.000000] Command line:
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[...]
[    0.022093] Booting paravirtualized kernel on Xen
[    0.022101] Xen version: 3.2-1

Xen 3.3 added a tool called PV-GRUB. It is a GRUB (legacy aka 0.97) built against MiniOS and bootable as a PV kernel. It works fine on a Xen 3.2.1. It is easy to use, I just set this in my config:

kernel = "/usr/local/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

However it decided to not load my config as defined in the config and simply show me that:

    GNU GRUB  version 0.97  (524288K lower / 0K upper memory)
       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]
grubdom>

The reason is quite easy:

grubdom> root (hd0)
 Filesystem type unknown, using whole disk

Some debugging later, the reason is known and fixed.

grubdom> root (hd0)
 Filesystem type is ext2fs, using whole disk
grubdom> kernel /boot/vmlinuz-2.6.27-rc5-amd64
grubdom> initrd /boot/initrd.img-2.6.27-rc5-amd64
grubdom> boot
block error -1 for op 2
close blk: backend at /local/domain/0/backend/vbd/23/51712
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.27-rc5-amd64 [...]
[    0.000000] Command line:
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[...]
[    0.022093] Booting paravirtualized kernel on Xen
[    0.022101] Xen version: 3.2-1

For those who did not yet got it: Xen upstream decided to drop non-PAE x86_32 support from Xen.

Tonight, I converted the XML-Security-C package from Stanford's internal Subversion repository to Git and packaged 1.4.0, but unfortunately I can't upload yet because Xalan needs to be updated to build against the latest Xerces-C. (I'd NMU, but I should not be taking on more things right now.) This is the first of the three current Shibboleth packages, and I hope to tackle OpenSAML and Shibboleth SP tomorrow or Wednesday. An Alioth project for Shibboleth packaging now exists and the pkg-shibboleth-devel mailing list is being created. I've requested Git space on git.debian.org, and as soon as that's set up, I'll push the three existing repositories. Shibboleth 2.0 packaging will add at least three more packages (OpenSAML 2.0, Shibboleth SP 2.0, and another XML support library), possibly four if we have to package log4shib, but I still hope to use the standard log4cpp. And then there will be another couple of packages once the group packages the Shibboleth IdP and WAYF, which I hope someone with more Java experience can tackle. One of the main goals for getting this set up is so that people other than me can do more of the work so that I can work on other things. Like, for example, krb5 packaging. Bastian Blank started using the Kerberos server packages and posted a whole flurry of bugs, a couple of which are (somewhat arguably) RC, and I'm trying to absorb that and fix the ones I can ASAP. OpenLDAP is in desperate need of attention, but I don't feel as much responsibility for that. I said going in that I'd only have time to work on it in fits and starts, and just because no one else has any time to work on it either doesn't change that. However, I do need to find time to post an RFH, since it really needs an active maintainer, and I'll at least find time to upload the upcoming 2.4.9 release before lenny freezes more, since the current version has a ton of bugs. Frank Lichtenheld has been doing a lot of the work on Lintian lately, which is wonderful, but I still have at least 10 bugs and patches that I want to deal with as soon as possible for it and get another upload out. We're well over my magic threshold of 100 bugs, and a lot of the stuff that's in the BTS just needs to be reviewed and committed. And then there's Policy, which is just about ready for an upload, and where I owe several responses on bugs and discussion threads. It's in the best shape right now of the stuff on my hot list, but there really should be a new upload in May, and preferrably in the first half. The current goal is to get the two other RC bugs against the Shibboleth packages dealt with in the next couple of days, then do a krb5 upload before the end of the week, and then Policy and Lintian are back at the top of the list, insofar as work lets me concentrate on anything else. BTW, I have to say, the more I use Git, the more I'm becoming a convert. I still think I'd rather use quilt if I have complex merges. Ironic, given that that's supposed to be Git's strength, but I have fifteen years of experience merging patches and I really understand how quilt works. That may mean I'll change my mind when I become even more familiar with Git. But for the typical package with a small number of divergences from upstream, I'm getting the hang of managing those on branches and I think I like it. I've yet to see a good reason to use rebase instead of merge, though, at least as part of a Debian packaging workflow. So either I'm still not getting something, or many of the people using Git are way more obsessive than I am about their branch merge graphs (which is weird, because I tend to be remarkably obsessive about things like that). I suppose if I ever get to using git bisect, that may change my mind.